-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Metricbeat] Adding cloudwatch metricset into AWS module #11798
[Metricbeat] Adding cloudwatch metricset into AWS module #11798
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we could take more advantage of the config parsing to simplify the code.
Also I am not sure if cloudwatch is generic enough to add its fields to the common structs for AWS config and metricset.
@jsoriano Yes you are right, cloudwatch should not be a part of aws module config. I will move it under cloudwatch metricset. Thank you! |
Most changed addressed, I will review again.
@jsoriano Thanks for the review and all the comments. Regarding to your comment about |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, I have added a couple of minor comments.
Could you please add a changelog entry? |
#11798 (comment) will be addressed in #12033 |
This PR is to add
cloudwatch
metricset intoaws
module for metricbeat.cloudwatch
metricset will read targetnamespace
from config file and query cloudwatch APIListMetrics
andGetMetricData
to get monitoring metrics for that specific namespace.Currently the config for cloudwatch metric looks like:
Fetch
function starts with checking config to see how manylistMetrics
entries are there as well as namespaces.This count as one
listMetrics
entry.readCloudwatchConfig
reads in the config and stores eachlistMetrics
entry andnamespace
if there's nometricname
information but onlynamespace
, likeFor
listMetricsTotal
, this is already good forcreateEvents
to use. But fornamespacesTotal
, we need to loop through each namespace and each region makingGetListMetrics
api call to getlistMetricsOutput
and use that forcreateEvents
function.In
createEvents
, there is a variable nameidentifier
, which is the combination of all dimensions inlistMetricsOutput
. For example,the identifier name is:
DBClusterIdentifier,Role
the identifier value is:
test1-cluster,READER
Some entries of
listMetricsOuput
does not havedimensions
and that's why there is aeventsNoIdentifier
to store events that came fromlistMetricsOutput
that doesn't havedimensions
/identifiers
.Please see more details in #11734